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SCIENTIFIC  REPORT 

During  the  period,  December  1,  1981  through  November  30, 
1982,  our  group  made  13  presentations  and  published  10  papers  (5 
in  journals  and  5  in  conference  proceedings) .  In  addition,  our 
group  published  5  technical  reports.  The  group  also  prepared  6 
papers  which  have  been  accepted  for  journal  publication.  A  list¬ 
ing  of  these  presentations  and  publications  is  provided  at  the 
end  of  this  report. 

The  group  devoted  its  efforts  to  four  areas  briefly 
described  in  the  next  section  of  this  report: 

1 .  Volumetric  Description  of  Objects  from  Multiple  views 

Volumetric  models  have  been  the  basis  of  numerous  three- 
dimensional  object  modelling  systems.  The  two  major  goals  pur¬ 
sued  in  our  work  are,  first,  to  lessen  the  dependence  on  feature 
point  measurements  in  a  structure  from  multiple  views  system,  and 
second,  to  develop  a  descriptive  three-dimensional  object 
representation  that  was  suitable  for  a  dynamic  process  of  volume 
refinement.  The  results  are  two  fold:  (i)  a  system  has  been 
developed  which  constructs  a  volumetric  structure  for  an  object 
from  a  sequence  of  occluding  contours,  and  (ii)  an  algorithm  has 
been  formulated  for  the  representation,  and  refinement  of  this 
structure. 

The  occluding  contours  with  viewpoint  specifications  from  a 
dynamic  image  are  analyzed  to  initially  form  and  continually 
update  a  description  of  the  three-dimensional  object  generating 
the  contours.  The  description  is  a  bounding  volume  for  the 
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object  and  is  successively  refined  to  yield  finer  approximations 
to  the  actual  object.  Of  course,  from  the  silhouettes  that  form 
the  occluding  contours  it  is  not  possible  to  resolve  certain 
kinds  of  concavities.  In  particular,  object  surface  points  for 
which  every  tangent  line  (in  the  tangent  plane,  any  line  that 
contains  the  given  surface  point)  also  intersects  the  object  at 
some  non-surface  point  cannot  be  resolved  using  silhouettes. 
However,  the  class  of  objects  that  can  be  described  exactly  is 
large,  and  in  fact,  the  object  surface  may  have  saddle  points  and 
holes. 

Clearly,  to  analyze  the  structure  of  objects  a  system  must 
provide  a  representation  scheme.  For  three-dimensional  objects 
many  different  schemes  have  been  proposed  and  used.  The  details 
of  the  representations  usually  are  determined  either  by  the  data 
acquisition  techniques  or  by  the  ultimate  application  of  the  sys¬ 
tem,  with  an  important  problem  being  the  development  of  methods 
for  transforming  between  structures  of  the  first  type  and  struc¬ 
tures  of  the  second  type.  The  volume  segment  representation 
described  in  our  work  has  been  developed  to  facilitate  the 
acquisition  of  three-dimensional  information  dynamic  images.  The 
main  attributes  of  the  volume  segment  representation  are  that  it 
is  easy  to  update  (as  required  by  the  continual  refinement), 
maintains  fine  surface  detail,  simplifies  the  point  inclusion 
test,  and  can  be  readily  transformed  into  a  surface  representa¬ 
tion. 

For  these  reasons  our  work  presented  provides  an  excellent 
basis  for  further  research.  In  particular,  the  work  is 
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appropriate  to  industrial  automation  applications.  For  example, 
selecting  one  of  several  parts  on  a  conveyor  using  the  views 
taken  from  several  cameras  fixed  along  the  line  of  travel,  or  in 
conjunction  with  a  manipulator  arm  that  could  successively  repo¬ 
sition  a  part  until  an  adequate  approximation  was  derived.  In 
such  applications  there  are  usually  fixed  sets  of  possible 
objects  from  which  an  unknown  object  must  be  recognized,  implying 
the  need  for  a  representation  scheme  suitable  for  creating  a 
library  of  possible  objects,  describing  the  unknown  sample  and 
matching  it  to  the  library  entries.  Further  research  will  be 
directed  toward  exploiting  the  methods  developed  here. 

2.  Contour  Registration  by  Shape  Specific  Points 

The  registration  of  contours  of  objects  is  often  an  impor¬ 
tant  aspect  in  many  image  processing  systems.  In  the  context  of 
shape  analysis,  for  example,  it  appears  as  a  crucial  step  in 
important  tasks  such  as  image  segmentation,  analysis  of  time- 
varying  imagery  and  binocular  perception  of  depth.  In  image  seg¬ 
mentation  shape  is  often  the  basis  for  the  detection  and  recogni¬ 
tion  of  patterns  such  as  lines,  edges  and  corners  and  more  gen¬ 
eral  Images  of  objects  such  as  characters.  In  the  analysis  of 
dynamic  scenes,  it  is  necessary  to  track  images  of  objects  from 
one  frame  to  the  next  in  a  sequence  of  time-varying  images;  and 
shape  analysis  is  an  important  tool  in  this  correspondence  pro¬ 
cess.  The  correspondence  of  features  is  also  a  central  issue  in 
stereomapping  where  it  is  necessary  to  identify  the  parts  of  two 
pictures,  taken  from  different  view  points,  that  represent  the 


same  object  in  the  actual  underlying  scene.  Thus  shape  matching 
has  been  motivated  by  a  variety  of  applications  and  has  led  to  a 
number  of  solutions.  Shape  analysis  techniques  depend  on  the 
model  used  to  represent  the  relevant  information  in  the  image  and 
the  success  of  these  techniques  depends  on  how  closely  the  model 
fits  the  actual  situation.  There  is  a  profusion  of  literature 
relating  to  shape  analysis  as  documented  in  C.3. 

Our  work  is  concerned  with  planar  shapes  and  presents  a  new 
approach  to  shape  matching  based  on  the  simple  observation  that 
matching  is  a  much  easier  task  when  shapes  are  properly 
registered.  Some  of  the  advantages  of  this  approach  stem  from 
the  fact  that  registration  allows  the  application  of  shape  match¬ 
ing  algorithms  (e.g.  template  matching)  that  cannot  otherwise  be 
applied  on  shapes  that  are  not  registered.  The  present  study 
thus  advocates  the  use  of  spatial  registration  before  matching 
and  describes  how  registration  of  two  planar  figures  can  be 
achieved  by  computing,  from  the  shape  of  each  figure,  points  that 
are  specific  to  the  shape  with  respect  to  transformations  of 
interest.  These  points  can  in  fact  be  considered  part  of  the 
shape  even  though  they  may  be  physically  apart  from  it.  The  cen¬ 
troid  is  one  example  of  such  points. 

Although  specific  or  "special"  points  and  lines  such  as  the 
centroid  and  the  major  and  minor  axes  have  often  been  used  in 
shape  analysis,  it  was  for  the  purpose  of  extracting  characteris¬ 
tic  features  to  describe  shapes  independently  of  the  coordinate 
system.  For  example,  Hu  represented  visual  patterns  such  as 
characters  by  lower-order  two-dimensional  moments  that  are 
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invariant  under  general  linear  transformations.  Then  a  simple 
clustering  procedure  was  used  to  classify  the  various  unknown 
patterns.  Thus  the  moments  method  transforms  patterns  into  a  set 
of  scalar  features  that  lose  the  spatial  organization  of  the  pat¬ 
terns. 

The  technique  reported  in  our  study  computes  shape  specific 
points  in  the  spatial  or  picture  domain  for  the  purpose  of  regis¬ 
tration.  The  present  technique  is  translation,  rotation  and 
scale  change  independent.  The  transformation  to  register  one 
figure  to  another  is  easily  recovered.  After  figures  are 
registered,  various  matching  algorithms  can  be  used  to  check  how 
closely  their  boundaries  match.  More  details  are  given  in  C.3. 

3.  Detection  of  Edges  Using  Range  Information 

Range  data  provide  an  important  source  of  3-D  shape  informa¬ 
tion.  Range  data  implicitly  contain  information  about  the  shape 
of  the  surface  of  objects  because  the  coordinates  of  points  on 
the  surface  of  these  objects  can  be  easily  recovered  from  them. 
Moreover,  they  can  be  used  to  extract  jump  boundaries  which 
correspond  to  occluding  boundaries  of  objects  in  a  scene,  and 
"edges"  which  are  points  that  lie  on  the  intersection  of  two 
regions  on  the  surface  with  significantly  different  parameters 
(e.g.  the  edge  between  two  visible  faces  of  a  cube) .  Jump  boun¬ 
daries  and  edges  are  important  cues  in  the  segmentation  process 
because  they  delineate  the  extent  of  surfaces.  These  boundaries 
and  edges  are  intrinsic  properties  of  the  surface  of  objects 
unlike  edges  in  "intensity"  images  derived  from  range  data.  For 
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example  one  may  create  an  image  by  using  a  range  coding  conven¬ 
tion  in  which  dark  =  near  and  light  =  far.  The  image  is  then 
enhanced  to  compensate  for  low  frequency  trends  present  in  sys¬ 
tems  which  have  shallow  line  of  sight.  Finally,  edges  are  com¬ 
puted  using  the  Sobel  operator.  In  our  work  we  are  concerned 
with  edges  that  actually  occur  on  the  surface  of  objects  in  the 
scene.'  Although  simple  methods  have  been  devised  that  are  suc¬ 
cessful  at  computing  jump  boundaries,  the  problem  of  finding 
edges  is  a  much  more  delicate  problem  which  has  not  received 
enough  attention.  Here,  we  are  mainly  interested  in  range  data 
obtained  from  sensors  such  as  the  one  at  SRI  International.  The 
major  problem  with  this  type  of  range  finder  is  that  the  accuracy 
of  the  measurements  depends  on  the  power  of  the  signal  that 
reaches  the  receiver.  The  accuracy  of  the  range  data  is  there¬ 
fore  dependent  on  the  parameters  of  the  system  (transmitted  beam 
power  and  receiver  variables)  and  on  the  characteristics  of  the 
target  (orientation  and  reflectance  of  its  surface  and  actual 
distance  from  the  sensor) .  System  parameters  and  target  charac¬ 
teristics  affect  accuracy  because  they  affect  the  strength  of  the 
signal  that  returns  to  the  sensor.  It  should  be  pointed  out  that 
we  do  not  restrict  ourselves  to  the  SRI  laser  sensor. 

Our  goal  in  this  study  is  to  design  and  analyze  a  procedure 
for  detecting  edges  using  range  information  that  has  low  sensi¬ 
tivity  to  noise.  We  also  want  to  relate  the  range  measurement 
accuracy  to  the  problem  of  detecting  edges.  The  input  to  this 
procedure  includes  range  data  and  a  model  for  range  measurement 
error.  Basically,  this  procedure  determines  the  best  partition 


of  a  neighborhood  of  each  point  in  the  scene  into  two  contiguous 
regions.  Planes  are  fitted  to  these  regions  and  a  measure  of  the 
goodness  of  fit  is  calculated.  The  measure  should  imbed  all  the 
knowledge  that  one  has  about  range  measurement  accuracy,  so  that 
the  resulting  value  can  be  used  to  select  the  "best"  partition. 
Subsequent  analysis  can  then  extract  significant  edges  from  the 
scene.  Although  we  are  fitting  planes  to  small  patches  on  the 
surface  of  objects,  we  are  not  looking  for  planar  surfaces.  We 
are  interested  in  determining  the  presence  or  absence  of  an  edge 
at  points  on  the  surface  of  objects.  Preliminary  results  are 
reported  in  B.3  and  C.4. 

4.  A  Normalized  Quadtree  Representation 

Quadtrees  are  hierarchical  data  structure  used  for  compact 
representations  of  two-dimensional  images.  A  quadtree  is  gen¬ 
erated  by  dividing  an  image  into  quadrants  and  repeatedly  subdi¬ 
viding  the  quadrants  into  sub-quadrants  until  each  quadrant  has 
uniform  color  (e.g.  '1*  or  'O'  in  a  binary  image).  The  root  of  a 
quadtree  corresponds  to  the  image  it  represents.  A  node  in  a 
quadtree  either  is  a  leaf  (terminal  node)  or  has  four  son-nodes 
(non-terminal  node) .  Each  son-node  is  associated  with  a  quadrant 
of  the  block  corresponding  to  its  father-node. 

The  advantage  of  the  quadtree  representation  for  images  is 
that  simple  and  well-developed  tree  traversal  algorithms  allow 
fast  execution  of  certain  operations  such  as  superposition  of  two 
images,  area  and  perimeter  calculation,  moments  computation,  etc. 
Other  researchers  have  shown  that  the  quadtree  representation  of 
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images  yields  substantial  data  compression  over  a  variety  of 
source  images.  In  their  experiments,  image  compression  ratio 
ranging  between  three  to  one  and  thirty-three  to  one  were  found, 
with  five  or  six  to  one  being  the  general  compression  factor. 

However,  the  quadtree  representation  has  certain  disadvan¬ 
tages.  The  quadtree  representation  of  an  object  in  an  image  is 
heavily  affected  by  its  location,  orientation  and  relative  size. 
A  small  change  in  these  parameters  will  generate  different  quad¬ 
trees.  To  eliminate  the  effect  due  to  the  translation  of  objects 
in  an  image,  one  defines  a  normal  form  for  quadtrees.  Assuming 
that  the  size  of  an  image  lies  between  and  2N,  the  image  is 
moved  around  a  region  of  size  2N+*  to  find  a  minimal  cost  quad¬ 
tree  in  terms  of  the  number  of  nodes.  Other  researchers  assert 
that  this  quadtree  representation  is  unique  for  any  image  over 
the  class  of  translations.  However,  the  problem  arising  from 
rotations  and  size  change  still  remains. 

In  our  research,  we  propose  a  representation  scheme,  the 
normalized  quadtree  representation,  which  is  invariant  to  object 
translation,  rotation  and  size  change.  Instead  of  generating  a 
quadtree  for  the  entire  image,  a  normalized  quadtree  is  generated 
for  each  object  in  the  image.  The  object  is  normalized  to  an 
object-centered  coordinate  system,  with  its  centroid  as  the  ori¬ 
gin  and  principal  axes  as  coordinate  axes,  and  then  scaled  to  a 
standard  size  (a  2N  x  2N  image).  In  this  way,  the  normalized 
quadtree  of  an  object  is  dependent  only  on  the  shape  of  the 
object,  but  not  affected  by  its  location,  orientation  or  relative 
size.  In  other  words,  the  normalized  quadtree  representation  can 
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be  utilized  as  a  shape  descriptor.  In  addition,  information 
related  to  the  size,  the  position  and  the  angle  of  the  major 
principal  axis  of  the  object  in  the  image  may  be  retained  ena¬ 
bling  reconstruction  of  the  object  as  it  appeared  in  the  image. 
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